Modeling data patterns and relationships with neural networks often requires dealing with non-convex optimization problems. However, a non-convex optimization problem may and often possesses multiple local minima. The difference between a local minimum and the global minimum in the non-convex optimization problem can be significant. Some optimization techniques resort to starting and restarting nonlinear optimization at random initial points in a hope to get sufficiently close to or even locate the global minimum (or optimum), although the chances of actually capturing the global minimum often are negligibly small due to a myriad of reasons including multiple local minima, numerical instability, etc. Another frequent concern is that a de-trending preprocessing may need to be performed over to-be-analyzed target data before it is used in modeling complex data patterns that may be mixed with an overall linear trend with neural networks. Indeed, if target data exhibits a strong linear trend, a neural-network based data analysis may not perform any better than a linear regression analysis.
Additionally, some techniques require significant subjective user involvements, generate solutions overly sensitive to noisy data, and frequently lead to incorrect solutions.
Based on the foregoing, there is a need for developing neural network data analysis techniques that mitigate these effects and difficulties.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.